Generate one exact contract across every Goa output - #3971

Community preview

v3.31.0-preview.1 is the published opt-in community preview for this branch. It will not be selected by go get ...@latest while a stable Goa release exists. Testers request it explicitly:

go get goa.design/goa/v3@v3.31.0-preview.1
go install goa.design/goa/v3/cmd/goa@v3.31.0-preview.1

Before regenerating, read UPGRADING.md. It explains why this work is needed, every known application and plugin break, coordinated client/server cases, migration steps, rollback, and what to include in a useful report.

The final stable version is deliberately undecided. This preview exists to test the corrected contracts with real services and plugins before deciding whether the final release can remain in v3 or requires a new major version. Please use this PR for preview-wide feedback, open a separate issue for a small reproducible bug, and use GitHub Discussions for migration or design questions.

Goa now chooses every generated package name, import, file, validation function, conversion function, and transport helper once for the complete generation run. Service, HTTP, gRPC, JSON-RPC, CLI, OpenAPI, example, and plugin output all use those same recorded choices.

The failure that started this work declared a validation function under one name and called another. Separate generator passes had created separate name scopes for the same Go package. Each pass saw a different set of conflicts, so each answer looked valid by itself while the combined source did not compile. The same split ownership affected union types, imports, transport conversions, examples, plugins, and repeated generation in one process.

UPGRADING.md is the public testing and migration guide. codegen/ARCHITECTURE.md is the complete internal design and generator-library contract. It includes the complete exported-API migration table, generated-source changes, mixed-version behavior, and rollback effects. This description highlights the contracts reviewers should understand first.

One generation run owns every answer

A generation run now has one saved plan: a typed record of the prepared design and every decision needed to write source.

1. Preparation plugins make their authorized design changes.
2. Goa records the prepared roots and creates one service plan for each root.
3. HTTP, gRPC, JSON-RPC, OpenAPI, examples, and factory plugins consume those exact service plans.
4. Each generated Go package collects all declarations, imports, and files that will belong to it.
5. Goa fixes names and import qualifiers once.
6. Templates write only those recorded choices.

Rendering cannot discover a declaration, allocate another name, rebuild transport analysis, or change the design. This keeps work that is known from the design inside generation instead of emitting runtime branches or parsing generator-made names later.

Ownership is explicit:

• codegen.Generation owns the prepared roots, generated packages, names, imports, and output paths for one run.
• service.Plan owns service types, endpoints, clients, errors, interceptors, views, unions, validators, and service conversions.
• HTTP owns HTTP bodies, parameters, multipart handling, SSE, WebSocket code, codecs, and validation.
• protobuf and gRPC own .proto declarations, protobuf Go names, presence, metadata, validation, and conversion.
• JSON-RPC consumes the exact shared HTTP plan, then owns JSON-RPC requests, responses, batches, errors, and SSE behavior.
• plugin planning owns names and declarations added by that plugin.
• file assembly combines valid same-file contributions and rejects conflicting or unsafe output paths before writing.

Repeated, concurrent, reversed-order, multi-root, multi-transport, and plugin-assisted runs now produce the same complete answer.

Generated contracts that become exact

OneOf unions

Copies of one authored OneOf share a declaration only when their emitted definitions match. Separately authored unions stay separate even when their branches happen to be equal. Public union names are exact; a true collision stops generation and asks the design author to give the declarations distinct TypeName values.

Compiler-created branch types use names such as ValueBranchText. HTTP union names describe the body that owns them, for example ValueRequestBody, ValueStreamingBody, ValueResponseBody, and ValueDetailedResponseBody. Relocated unions are written by their owning package in unions.go.

Generated branch fields are private. Callers use New..., Set..., As..., Kind, and Validate. Selecting a branch replaces the previous selection, and a failed JSON decode leaves the prior valid value unchanged.

Required unions reject no selection, typed nil wrappers, and selected nil message, bytes, or Any values. A selected nonnil empty message remains valid. JSON and protobuf union data do not change.

Protobuf and gRPC

Required singular booleans, numbers, strings, enums, bytes, and their aliases use proto3 presence. Generated Go scalar fields become pointers; bytes and bytes aliases remain []byte; Any and other messages remain pointers. Goa service fields keep their existing value layout. Protobuf field numbers and binary tags do not change.

Generated clients and servers validate protobuf messages before converting them. Omitted required fields now return precise validation errors instead of silently becoming service zero values. Explicit false, 0, empty string, empty bytes, and protobuf null remain valid when supplied.

Defaults now follow presence. An absent protobuf input receives its authored default. An explicit zero, empty bytes, or protobuf null remains explicit. Service-to-protobuf conversion never adds defaults; it sends exactly what service code returned.

Each selected gRPC result view has its own conversion and validation. Fields omitted by that view are not required. Dynamic server streams send the selected view before the first message, and clients use it before decoding. Default validators keep names such as ValidateShowResponse; another view uses a stable name such as ValidateShowResponseTiny. Equal validators share one declaration instead of receiving discovery-order suffixes.

Repeated and map wrappers no longer claim protobuf can distinguish omitted from empty. Nil map-value wrappers decode as empty collections instead of panicking, while authored length and item rules still run. Validators that provably do nothing are not emitted.

gRPC metadata conversion uses the designed type. Bytes use their actual string contents rather than Go slice display text, floating-point values use the designed width, and response encoders use the real result variable.

Generation checks protoc-gen-go v1.36.12 and protoc-gen-go-grpc v1.6.2 before files are written. These are the exact tools covered by the generated-module tests.

HTTP

Incoming JSON arrays declared with ArrayOfRequired use pointer elements for primitives and primitive aliases so [null] is rejected. Valid JSON converts to the same service value slices, and outgoing bodies remain value slices.

Multipart decoders now fill the generated request body, validate it, and only then build the service payload. Nested validation errors keep their complete field and array-index paths. Exclusive maximum validation rejects the maximum itself and values above it.

A map assigned to the complete query string now reads raw keys such as ?a=1&b=2, matching generated clients. Float query values use Go's shortest round-trip text. Generated clients close bodies they fully consume and return read and close errors; a deliberately returned raw body remains open for the caller.

SSE writes primitive values as raw event text, distinguishes an omitted optional value from a present empty string, returns write and flush failures, and decodes retry values into the designed integer type. A variable result view is fixed before the first event; unknown or changing views fail precisely.

An empty successful WebSocket stream now performs the upgrade, sends a normal close frame, and closes once instead of returning before the handshake.

JSON-RPC

JSON-RPC now supports two honest method shapes: one request and response over HTTP, or one request followed by server results over explicit server-sent events. Design validation rejects client streams, bidirectional streams, WebSocket streaming, server streams without ServerSentEvents(), and methods that define both Result and StreamingResult.

Generated SSE implementations use the transport-independent service stream methods Send, SendWithContext, and Close. Clients use Recv or RecvWithContext. Each result is a JSON-RPC notification. A request with an ID ends with one terminal response: result: null for success or the returned JSON-RPC error. A notification receives no terminal response.

Request handling now follows JSON-RPC 2.0 for omitted, null, empty-string, string, and numeric IDs; invalid objects; leading whitespace; empty and mixed batches; notifications; explicit result: null; invalid parameters; internal failures; acceptable JSON/SSE media types; and rejection of streams inside batches. Clients reject unknown event names and notifications for another method instead of silently skipping them. Body reads, closes, and batch writes return their failures.

A caller-selected view uses this method result inside JSON-RPC's standard top-level result member:

{
  "view": "detailed",
  "body": { "...": "..." }
}

This envelope is generated only when the caller chooses among views. Fixed-view and unviewed methods retain their body shape. The envelope is valid generic JSON-RPC; a protocol layered on JSON-RPC, including MCP, must still use that protocol's required result schema.

API errors, interceptors, CLI, examples, and OpenAPI

An API-level error is a reusable definition, not an error returned by every endpoint. A service or method selects it with name-only Error("busy"), which preserves its type, validation, defaults, description, and Temporary, Timeout, and Fault settings. Supplying another argument defines a separate local error.

Generated interceptor information changes from *LoggingInfo, a pointer to a public struct with private fields, to the read-only LoggingInfo interface with the same public accessors. Goa emits a private implementation specialized for the exact method and call kind, so payload, result, send, and receive accessors do not inspect method names or switch on runtime types.

Generated commands execute the endpoint, receive streams, print values, and return endpoint, stream, output, and close errors. gRPC complete-message flags decode protobuf JSON. Example values belong to the declaration that authored them, so an earlier example cannot consume shared random state and alter a later one.

OpenAPI now emits consistent base64 byte examples, empty security scope arrays rather than JSON null, independent server-variable values, designed descriptions, correctly filtered security definitions and examples, and schemas that match selected views and SSE data.

Plugin compatibility

The released four-argument registration functions, Genfunc, replaceable Generators, and the exported Service, Transport, OpenAPI, and Example functions remain available. Released callback ordering and repeated callback names remain supported. Built-in functions join the shared plan; external Genfunc values run after names are final.

Common generated-name fields used by existing templates remain, including MountHandler, HandlerInit, constructors, codecs, validators, multipart helpers, SSE names, WebSocket names, and gRPC names. Plugins that edit ordinary values or files should largely continue to work.

A plugin that adds a package declaration or chooses a generated name must use PluginFactory and declare it during Plugin.Plan. A preparation plugin that adds services must attach them to the owning root and call EvaluateAttachedServices. Public helpers that manually ran plugin callbacks or rebuilt private service or transport analysis are removed because they would create a second set of decisions.

Several exported planning and template-data structures changed. Some gained declaration records or private state, some no longer compare with ==, and positional literals must become named literals. codegen/ARCHITECTURE.md lists every changed exported API and its replacement.

Upgrade, mixed versions, and rollback

Updating the Goa module does not change an already compiled program. Changes take effect when code is regenerated. There is no persisted-data migration.

Regenerate the entire gen tree together; declarations and callers from different generations are not compatible. goa example preserves handwritten starter files, so update those separately.

Coordinate client and server deployment and rollback for:

• caller-selected JSON-RPC views;
• JSON-RPC server-sent-event streams;
• dynamic caller-selected gRPC streams;
• viewed gRPC methods whose selected view omits fields used by the old default conversion; and
• HTTP SSE streams with optional primitive data.

Required protobuf presence keeps the binary schema compatible, but an old client cannot prove that a required zero or empty primitive was present. A new server can reject that omission. Regenerate both sides where required zero or empty values matter.

Other intentional source changes affect direct union field access, interceptor info pointers, multipart decoder signatures, incoming ArrayOfRequired transport literals, protobuf scalar fields, direct calls to removed empty validators or combined conversions, generated command starters, gRPC protobuf-JSON CLI scripts, and JSON-RPC WebSocket APIs.

JSON-RPC WebSocket has no compatibility mode. Migrate those methods to unary JSON-RPC over HTTP, explicit JSON-RPC SSE server streaming, ordinary HTTP WebSocket, or gRPC before upgrading.

The detailed table in codegen/ARCHITECTURE.md also covers compact HTTP float text, gRPC metadata text, whole-query maps, exact output-path failures, stricter design validation, OpenAPI snapshot changes, and every exported generator-library change.

Review this first

1. codegen/ARCHITECTURE.md for the lifecycle, ownership rules, and complete migration contract.
2. codegen/generation.go, codegen/generated_types.go, and codegen/generator/plan.go for run and package ownership.
3. codegen/service/plan.go and codegen/service/generated_package.go for service declarations and names.
4. codegen/union.go, codegen/validation.go, and the service/HTTP union templates for exclusive union behavior.
5. grpc/codegen/protobuf_catalog.go, grpc/codegen/service_data.go, and codegen/go_transform.go for protobuf presence, defaults, views, and conversions.
6. jsonrpc/types.go and jsonrpc/codegen/{plan,server,client,sse,viewed_result}.go for protocol and streaming behavior.
7. codegen/generator/plugin_public_integration_test.go for released plugin compatibility.

Validation performed

• go test ./... passes on current commit 26fd7ec677c2, including generated temporary modules and all service, HTTP, gRPC, JSON-RPC, OpenAPI, example, and plugin packages.
• make lint reports 0 issues, all GitHub checks pass, and there are no pending review comments.
• Focused generated-runtime tests cover exclusive unions, required OneOf values, required protobuf primitives and aliases, protobuf defaults, collection wrappers, selected gRPC views, JSON-RPC protocol rules, JSON-RPC viewed unary/SSE results, and released plugin compatibility.
• A fresh AURA generation reduced private gRPC helpers from 1,945 to 1,349 and reduced 366 exact duplicate copies to zero. All 1,070 generated gRPC validators and all 1,349 generated gRPC conversions are distinct within their packages.
• Flows commit a81b5b1665e8 pins this Goa commit and Goa-AI 1ba7a23d3da5; two regenerations were identical, and codegen checks, tests, lint, and builds pass.
• Goa-AI commit 1ba7a23d3da5 pins this Goa commit; its race tests, lint, build, quickstart, MCP fixtures, Redis registry integration, and Mongo run-log migration integration pass.
• All 19 modules in the Goa examples repository pin this commit at 213ae3f2e422. Two complete regeneration passes were identical, every module builds and tests, and live HTTP, gRPC, validation, viewed-result, and Cellar behavior pass.
• The Goa plugins repository pins this commit at 3a4b508b97d4. Generation and all plugin builds pass twice with no generated, golden, protobuf, OpenAPI, or public plugin API change from the final pin.
• AURA commit ed15cf06fb3 contains current AURA main and pins the exact Goa, Goa-AI, and Flows commits above. Repeated full generation is byte-for-byte stable, the independent regeneration check passes, every generated package compiles, 158 race-enabled test packages pass, full lint reports 0 issues, and every service builds.
• Every changed Goa golden was compared with the contract that owns it. AURA OpenAPI routes and structural schemas were also compared with main: the route set is unchanged, and structural changes are limited to the intended attachment API and date-format contracts. The large one-time example-text diff comes from making example ownership deterministic; repeated generation produces no further change.
• An exported-API comparison against origin/v3 reports 109 incompatible declarations. Every group and its migration is documented in codegen/ARCHITECTURE.md and summarized above.

All downstream repositories use the remote branch fix/goa-generation-plan. This pull request remains draft for review, not because downstream verification is incomplete.